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ABSTRACT 


sequential  design  for  estimating  the  percentiles  of  a  quantal  response 
curve  is  proposed.  Its  updating  rule  is  based  on  an  efficient  summary  of  all 
the  data  available  via  a  parametric  model.  Its  efficiency  in  terms  of  saving 
the  number  of  runs  and  its  robustness  against  the  distributional  assumption 
are  demonstrated  heuristically  and  in  a  simulation  study.  A  linear 
approximation  to  the  "logit-MLE"  version  of  the  proposed  sequential  design  is 
shown  to  be  equivalent  to  an  asymptotically  optimal  stochastic  approximation 
method,  thereby  providing  a  large  sample  justification.  For  sample  size 
between  12  and  35,  the  simulation  study  shows  that  the  "logit-MLE"  version  of 
the  general  sequential  procedure  substantially  outperforms  an  adaptive  (and 
asymptotically  optimal)  version  of  the  Robbins-Monro  method,  which  in  turn 
outperforms  the  nonadaptive  Robbins-Monro  and  Up-and-Down  methods.  A 
nonparametric  sequential  design,  via  the  Spearman-Karber  estimator,  for 
estimating  the  median  is  also  proposed.^- 
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SIGNIFICANCE  AND  EXPLANATION 


In  many  physical  or  biological  experiments  with  binary  response  a  quantal 
response  curve  is  assumed  to  relate  the  probability  of  response  to  the 
corresponding  level  of  the  stimulus  variable.  To  estimate  the  percentiles  of 
the  quantal  response  curve  efficiently,  a  sequential  design  is  often  used  in 
practice.  We  propose  a  new  class  of  sequential  designs  with  updating  rules 
based  on  an  efficient  summary  of  all  data  available  via  a  parametric  model. 
This  method  is  shown  to  be  asymptotically  as  good  as  the  optimal  stochastic 
approximation  method.  More  importantly,  its  finite  sample  performance  in  a 
simulation  study  is  better  than  the  latter  method.  Specifically,  the 
percentage  of  runs  saved  by  using  our  method  ranges  from  25%  to  60%. 


The  responsibility  for  the  wording  and  views  expressed  in  this,  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 


EFFICIENT  MODEL-BASED  SEQUENTIAL  DESIGNS 
FOR  SENSITIVITY  EXPERIMENTS 

C.  F.  Jeff  Wu* 

1 .  Introduction 

A  sensitivity  experiment  is  characterized  by  a  response  curve  that 
relates  the  stimulus  level  applied  to  an  experimental  subject  to  the 
probability  o  f  response.  The  outcome  of  the  experiment  is  assumed 
dichotomous,  response  or  nonresponse.  This  situation  arises  in  many  fielos 
of  research.  In  testing  the  strength  of  materials,  the  stimulus  level  may  be 
the  level  of  impact  energy  applied  to  a  piece  of  material,  and  the  response 
is  either  "fail*  or  anot  fail*  (Uetherill,  1963).  In  testing  explosives,  the 
stimulus  level  may  be  the  height  from  which  a  weight  is  dropped  or  the 
pressure  directly  applied  to  the  explosive,  and  the  response  is  'explode*  or 
*not  explode*  (Dixon  and  Mood,  1948).  In  biological  assays  a  test  animal 
survives  or  not  at  a  given  dose  level  (Finney,  1978).  In  psycho-physical 
research  the  probability  of  detecting  a  stimulus  is  related  to  its  intensity 
level  (Rose  et  al.,  1970).  In  educational  testing,  one  may  want  to  study  the 
•item  characteristic  curve*  that  relates  the  difficulty  level  of  the  test 
item  to  the  probability  of  ‘right*  or  ‘wrong*  answer  (Lord,  1971). 

Our  main  interest  is  in  estimating  the  percentiles  of  the  response  curve 
F(x),  which  is  the  probability  of  response  for  a  given  stimulus  level  x.  The 
lOOp  percentile  Lp  is  defined  as 

(I)  F(Lp)  ■  p. 

For  simplicity  we  assume  F  is  monotone  increasing  and  continuous.  The  median 
of  F,  Lq  5,  is  the  most  commonly  used  measure  of  a  characteristic  of  the 
response  curve.  In  some  situations  estimating  Los  is  of  intrinsic  interest, 
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but  more  often  it  it  because  L0  *.  it  eat/  to  ettimate.  In  quality  assurance 
it  may  be  more  interesting  to  study  the  extreme  percentiles,  e.g.,  to  find 
the  impact  energy  level  that  results  in  the  failure  of  material  for  at  most 
10 V.  of  the  time.  On  the  other  hand  Lq^  may  be  more  relevant  in  explosive 
research. 

In  this  paper  we  will  present  some  new  sequential  designs  for  the 
efficient  estimation  of  for  small  or  moderate  sized  experiments.  As  will 
be  explained  later,  our  method  is  more  appropriate  for  0.2  (  p  i  0.8.  We 
consider  sequential  designs  in  such  a  way  that  all  the  information  in  the 
previous  experimentation  can  be  utilized  in  a  most  efficient  manner  for 
suggesting  how  the  next  experiment  should  be  performed.  When  the 
experimental  runs  are  very  expensive,  the  saving  of  a  few  runs  by  an 
efficient  design  outweighs  the  extra  pains  taken  in  designing  a  sequential 
experiment.  The  sequential  nature  of  the  design  requires  quick  responses  so 
that  the  experiment  will  not  be  unduly  prolonged.  It  is  suitable,  for 
example,  when  the  experimental  facility  is  limited  so  that  experimental  runs 
must  be  performed  one  after  another.  Many  biological  experiments  that 
involve  inexpensive  animals  and  slow  responses  have  to  be  ruled  out.  A  key 
element  of  our  sequential  scheme  is  the  efficient  summary  of  all  available 
information  for  suggesting  the  next  design.  This  requires  a  certain  degree 
of  computing.  As  computing  becomes  cheaper  and  more  personalized,  the  cost 
of  automating  an  experimental  design  will  be  less.  By  taking  all  the  factors 
into  account,  our  method  is  more  appropriate  for  expensive  experiments  with 
short  response  time,  which  are  more  often  encountered  in  engineering 
research.  In  educational  or  psychological  testings,  if  a  test  has  to  be 
repeated  routinely  on  many  subjects,  it  pays  off  to  automate  the  design  and 
to  look  for  the  most  efficient  ones  (in  terms  of  reducing  the  number  of  test 
items) . 

In  the  next  section  we  shall  review  two  nonparametr ic  sequential  designs 
and  point  out  their  inappropriateness  for  the  scenarios  described  above.  Our 
approach  is  to  assume  a  parametric  model  for  the  response  curve  and  estimate 
efficiently  the  relevant  parameters  in  the  model  based  on  all  the  data 


available.  An  eat i mated  Quanta!  response  curve  <EQRC>  is  conatructed  through 
the  current  eatinate  o  f  the  parameters  and  the  next  design  point  ia 
determined  from  the  EQRC.  Some  heuristic  and  theoretical  justifications  are 
provided  for  the  methodology.  In  particular,  a  linear  approximation  to  the 
■logit-MLE*  version  of  the  EQRC  approach  ia  shown  to  be  equivalent  to  an 
asymptotically  optimal  stochastic  approximation  method.  The  Monte  Carlo 
results  of  the  last  section  indicate  that  the  EQRC  approach  is  not  sensitive 
to  distributional  assumptions  for  estimating  Lq  j  and  Lg  7g  and  substantially 
outperforms  the  Robbins-Monro  stochastic  approximation  method  and  the 
Up-and-Down  method,  including  an  adaptive  <and  asymptotically  optimal) 
Robbins-Monro  method.  For  the  particular  simulation  experiment,  our  method 
results  in  saving  25%  to  d0%  of  the  total  number  of  runs  required  by  its 
nearest  competitors.  The  empirical  study  also  reveals  that  a  mild  degree  of 
truncation  is  needed  for  both  our  method  and  the  adaptive  Robbins-Monro 
method  to  perform  stably.  A  good  guess  of  the  initial  design  and  the  step 
size  is  more  critical  to  the  performance  of  the  Robbins-Monro  and  Up-and-Down 
methods  than  the  parametric  assumption  is  to  our  method.  For  details  see 
Section  6.  A  nonpar ametr ic  sequential  design  for  estimating  the  median  LgS 
is  proposed  via  the  Spearman-Karber  estimator.  Its  limitations  are 
discussed. 


2.  Review  and  criticism  of  the  Stochastic  Approximation  method  and  ihe 
Up-and-Down  method. 

The  Stochastic  Approximation  method  and  the  Up-and-Down  method  are  two 
most  commonly  used  nonparametr ic  sequential  designs  for  quantal  response 
problems. 

Stochastic  Approximat ion  Method  (Robbins  and  Monro,  1951): 

Let  yR  «  1  or  0  as  the  nth  experiment  results  in  a  response  or 
nonresponse.  For  estimting  L^,  the  stimulus  level  xn+j  of  the  (n»i)th  run  is 
chosen  according  to 


x  -  ^  (y  -p) i 
n  n  *n  r 


-3- 


According  to  the  results  of  Chung  <1954),  Hodges  and  Lehmann  (1955)  and  Sacks 

(1958),  the  optimal  choice  of  c  in  <1)  is  <F'<L  >>“I.  Procedure  <2>  with 

P 

this  choice  of  c  is  asymptotically  consistent  and  fully  efficient,  i.e.,  xfj 

— •  Lp  a.s.  and  its  asymptotic  variance  is  minimized.  The  small  sample 

behavior  of  <2)  depends  very  much  on  a  good  starting  value  Xj  (Uetherill, 

1963).  Ideally  Xj  should  be  close  to  Lp.  A  good  guess  of  the  optimal 

constant  c  may  also  be  hard  to  come  by  since  in  most  practical  situations  the 

experimenters  have  little  idea  about  the  slope  of  F  at  L_.  Poor  choice  of  c 

P 

and  Xj  will  make  <2)  an  inefficient  procedure  for  small  and  even  moderate 
samples.  The  Stochastic  Approximation  method  has  been  used  more  effectively 
in  on-line  estimation  wherein  a  large  number  of  data  have  to  be  processed 
quickly. 

To  achieve  minimal  asymptotic  variance,  it  is  necessary  to  estimate  the 
slope  F'tLp).  One  such  estimator-is  the  regression  slope  of  y-t  over  xjP 


:  .EVi -V 

pn  n - 7 

1  <*! -*„> 


in 

ifi- 


The  procedure  <2)  with  c  ■  Pn*  is  aptly  called  a  stochastic  Newton-Raphson 
method  by  Anbar  (1978),  since  it  can  be  viewed  as  a  method  of  solving  the 
equation  F<x>  *  p  by  the  tangential  approximation  to  F  at  xft  with  F<xn> 
replaced  by  yft  and  F'<xn>  by  pn.  Under  various  regularity  conditions,  Anbar 
(1978)  and  Lai  and  Robbins  (1981)  proved  that  pn  — »  F'<Lp>  a.s.  and  that  the 
procedure  <2a)  is  asymptotical ly  optimal, 


*n  “  TT  <yn  “  >n  ln  <3) 
n^n 


When  n  is  small  or  the  current  guess  xR  is  on  the  tails  of  the  response 
curve,  may  behave  erratically.  Since  the  tails  of  the  response  curve  are 

A  . 

flat,  pn  with  (Xj )v  located  on  either  tail  tends  to  be  closer  to  zero,  thus 


making  the  adjustment  from  xR  to  xr4]  in  <2a)  unreasonably  large.  This 
happens  when  the  initial  guess  is  poor  for  estimating  the  median  or  when  the 
initial  design  takes  too  few  points  from  the  middle  part  of  the  response 
curve  for  estimating  the  extreme  percentiles.  To  remedy  this,  we  propose  to 
truncate  pn  ,  that  is,  to  use  max<min<pn  ,c),-c>  instead  of  pn  for  some 
positive  constant  c.  The  simulation  study  of  Section  6  shows  that  there  is 
considerable  improvement  in  using  this  truncated  version  of  <2a>.  Anbar 
(1978)  and  Lai  and  Robbins  (1981)  considered  truncating  p„  instead  of  p~* 
mainly  for  technical  reasons. 

Uo-and-Pown  method  (Dixon  and  Mood,  1948) t 


(4) 


V»1 


-.-"'■‘I’ 


The  method  works  only  for  Lq>5.  It  is  very  simple  to  implement  but,  for 
small  or  moderate  samples,  its  performance  depends  very  much  on  a  good  guess 
of  Xj  and  A.  Unless  the  step  size  A  is  made  adaptive,  the  large  sample 
property  of  xR  can  not  be  studied.  Its  empirical  performance  is  usually  not 
as  good  as  the  Stochastic  Approximation  method.  See  Uetherill  (1963)  and 
Section  6  of  the  present  paper.  Some  modifications  of  the  two  methods  can  be 
found  in  Uetherill  (1963,  1966). 

Both  methods  are  'Markovian*  in  that  the  choice  of  the  next  run  depends 
sensibly  on  the  outcome  of  the  current  one.  Their  simpl ic i ty  was  a  positive 
factor  when  inexpensive  computing  was  not  accessible.  Their  main 
disadvantages  are:  (a)  The  updating  rules  (2)  and  (4)  do  not'make  use  of  all 
the  data  available  in  an  efficient  way,  and  thus  making  the  choice  of  step 
size  less  flexible,  (b)  Their  small  sample  behavior  depends  on  a  good  choice 
of  the  relevant  constants  in  (2)  and  (4),  which  in  turn  depends  on  the 
experimenter's  knowledge  of  the  unknown  response  curve  F.  For  small  or 
moderate  sized  experiments  with  expensive  runs,  inefficiency  and  lack  of 


robustness  can  be  quite  serious.  Large  sample  properties  seem  quite 
irrelevant  in  this  context. 

The  sequential  design  method  proposed  in  the  'following  section  is  quite 
different  from  <2)  and  (4)  with  respect  to  the  shortcomings  (a)  and  <b>. 

3.  A  class  of  seouent ial  designs  based  on  the  est imated  quanta!  response 
curve . 

A 

Ideally  we  would  like  to  have  a  good  estimate  Ffl  of  the  whole  curve  F, 
from  which  the  next  design  point  xR4j  is  chosen  to  be  its  lOOp  percentile, 

A  A 

i.e.,  Ffj(xn+j>  =  p.  A  (smooth)  nonparametric  estimate  Fn  of  F  is  not 

A 

feasible  since  it  requires  a  large  number  of  observations  for  Ffl  to  be  a  good 
estimate.  A  natural  approach  for  smalt  sample  problems  is  to  assume  a 
parametric  model 


F(x)  *»  F(xJ®>,  F  is  continuous  in  x, 

1 im  F(xU)  *  0,  lira  F<x|«>  «  1 . 
x— ♦-«  x-*« 

The  general  recipe  of  our  sequential  design  procedure  for  estimating  L^  is: 

A  ^  n 

<i)  find  an  efficient  estimate  $n  *  •<<yi,Xj)j)  of  4, 

<5) 

A  I  A 

<ii)  define  the  est imated  ouantal  response  curve  (EQRC)  Fn<x)  «  F<x|On), 

A 

and  choose  the  next  design  xnfj  s.t.  Fft<xn+j)  =  p. 


Recall  that  y  {  «  I  or  0  is  the  response  or  nonresponse  at  level  Xj 


In  general  the  choice  of  the  parametric  model  F<*|#>  should  reflect  the 
experimenter's  knowledge  of  the  problem,  if  there  is  any.  Given  this  model, 
were  there  a  reliable  prior  on  9,  a  Bayesian  approach  for  estimating  9  would 
be  appropriate.  In  the  absence  of  such  information  <a  more  typical  situation 
in  practice),  we  suggest  to  use  the  logit  model 

«>  F<x|.)  -  »  >  0.  «  -  <«,»> 

A  A 

and  the  maximum  likelihood  estimator  (MLE)  <«,*)  of  (a,X>.  For  (6), 
L.  “  "  1)  and  its  MLE  L_  «  a  -  i  ln(£  -  1>.  Note  Ln  e  ■  a. 

p  A  p  p  £  p  B.3 

The  main  reason  for  preferring  logit  to  its  competitor,  the  probit 
mode  1 , 


<7) 


G(x|«)  »  —1 —  [X 
1 


(z-u)2 

"2bT~ 

e  £0  dz,  •  ■  (p,e>,  c  >  0, 


2s  e 


is  computational  ease.  It  is  well  known  that  the  logit,  the  probit  and  other 
parametric  models  like  the  angular  and  the  linear  curves  agree  very  closely 
in  the  range  0.2  to  0.8  <Cox,  1970,  Table  2.1).  We  do  not  see  any  advantage 
in  using  the  probit  over  the  logit,  although  it  is  a  legitimate  choice.  It 
is  rarely  the  case  that  a  parametric  quantal  response  model  be  justifiable  on 
biological  or  physical  grounds.  The  successful  use  in  practice  of  the 
parametric  approach  for  quantal  response  problems  is  mainly  due  to  this  key 
fact  that  the  parametric  curves  (after  adjusted  for  location  and  scale)  agree 
very  closely  in  a  wide  range  of  p  values.  For  p  outside  [0.1,0.93  the 
percentiles  for  different  parametric  models  vary  greatly.  Therefore  we  can 
not  recommend  our  procedure  (5)  for  extreme  p  values  unless  there  is  a  good 
reason  to  believe  in  the  particular  model.  One  may  argue  that  the  Stochastic 
Approximation  method,  being  nonpar arae trie,  still  works  for  the  extreme  tails. 
This  is  only  so  for  very  1  aroe  samples.  For  instance,  the  method  makes  on 


the  average  nine  negative  moves  tor  each  positive  move  in  the  neighborhood  of 


l0>9.  Instead  of  'straddling*  Lg  9,  the  sequence  makes  tar  too  man/  moves  in 
one  direction.  This  explains  the  much  poorer  empirical  pertormance  ot  the 
Stochastic  Approximation  method  (2)  even  tor  moderate))'  extreme  tails  like 
L0.75  <w,therm>  1963,  \6 ). 

A 

The  next  issue  is  the  choice  ot  etticient  estimator  <L  in  (5)  (i).  The 

n 

minimum  logit  chi-square  method  (Berkson,  1955)  is  not  suitable  tor  the  kind 
ot  data  generated  by  a  sequential  procedure  like  (5),  especially  tor  small  or 
moderate  samples.  This  is  because  there  are  tew,  and  typically  only  one  *•* 
two,  observations  at  a  given  x  level  to  make  the  minimum  logit  chi-squ 
work.  Unless  we  restrict  the  search  ot  design  levels  to  a  small  number  o- 
levels,  the  situation  will  not  be  much  changed.  The  same  remark  applies 
the  minimum  modified  chi-square  method,  and  to  a  lesser  extent,  to 
minimum  chi-square  method.  The  maximum  likelihood  estimate  of  <a,X>  in  <6> 
is  obtained  by  iteratively  solving  the  equations 

,|,  F<»,  |«.»>  >  p, . 

(8) 

n  .  n 

£  x jF<Xj |«,X>  »  p'jXj , 

where  F(x|«,X)  ■  <1  ♦  e"i<x"flc)  )”* .  Th  MLE  is  a  function  of  the  sufficient 
statistics  <2>j  •  JIx ■  x j ) .  It  is  asymptotically  fully  efficient  given  the 
right  model  and  is  an  efficient  summary  of  all  the  information  available  in 
small  samples.  Unless  there  is  a  reliable  prior  on  9  so  that  a  Bayesian 
approach  (Freeman,  1970;  Tsutakawa,  1972;  Owen,  1975;  Leonard,  1982)  becomes 
effective,  it  might  be  hard  to  beat  the  MLE  for  small  samples* 

A 

Given  an  efficient  estimate  0n,  the  sequential  design  (5)  makes  full  use 
of  all  the  information  available  and  the  step  size  *n4j_xn  is  more  flexible, 
i.e.,  it  is  capable  of  making  large  or  small  adjustment  as  the  situation 
calls  for.  Its  only  *ad  hockery*  is  in  the  logit  assumption.  As  argued 
before,  the  assumption  is  quite  robust  for  0.2  4  P  4  0.8.  It  will  be  further 
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supported  in  the  empirical  stud)'  of  \6. 

For  the  implementation  of  procedure  (5),  it  is  important  to  know  when 
the  MLE  exists.  To  avoid  trivialities,  assume  there  are  at  least  two 
distinct  Xj's.  It  is  known  (Silvapulle,  1981)  that  the  MLE  of  the  ‘linear* 
parameters  Q,Xa)  in  the  logit  model  <6>  exists  uniquely  iff 


<9.1  > 
or 

<9. 2) 
or 

(9.3) 


<x*in,  A  <x".  ,  x"  >  is  non-empty 

min  max  min’  max 


x  •  <  x 

min  min 


x  <  x 
max  max 


x  <  x  . 
min  min 


x  <  x 
max  max 


"h*rr  xmax(min)  “  m*x<min>  {xi s  yi  “  *>»  xmax<min>  *  m*x<min)  <xjS  y,  -  0). 
The  same  result  holds  for  more  general  distributions  F  including  the  probit 

model  (7).  See  Silvapulle  <1981,  Theorem  (iii>).  It  is  easy  to  see  that 

<9) ,  once  satisfied,  is  always  satisfied  by  the  addition  of  more 

obver sat  ions.  The  change  from  xn  to  xR+1  via  the  logit-WL~  method  may  be 
unduly  large  when  the  problem  is  *i11-posed.*  It  happens  when  the  data 
configuration  <x  j  ,Xj  >”  is  such  that  the  first  time  the  condition  (9)  is 
satisfied  is  n  or  n-k  with  very  small  k,  that  is,  the  existence  and 

uniqueness  of  MLE  has  only  been  guaranteed  in  the  current  or  last  few  runs. 

Ue  propose  a  truncated  version  as  follows.  Define  dn  as  the  solution  of 

d-  *  A  _  1  •  A  A 

x..,  ■  x.  -  —  <y„  -  p),  where  x_.,  *  ec.  -  ln(p  -1)  and  (or  ,50  is  the 
n*i  n  n  n  n+i  n  n  r  n’  n 

solution  of  <8).  The  <n«l)*h  design  level  is  chosen  to  »e 


<10> 


max(-d,  min(dn,d)>, 


where  d  is  a  given  positive  constant.  The  procedure  (10)  is  shown  in  Section 
6  to  perform  extremely  well  over  a  broad  range  of  the  truncation  constant  d. 
For  very  large  d,  like  600,  which  is  equivalent  to  almost  no  truncation,  (10) 
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does  not  perform  very  stably. 

The  question  of  how  to  choose,  say,  the  first  ten  runs  is  very  difficult 
unless  some  prior  knowledge  is  available.  It  may  be  done  in  an  ad  hoc  manner 
aided  with  experience,  or  by  the  Stochastic  Approximation  procedure  (2)  with 
a  reasonable  guess  of  Xj  and  a  slightly  1 aroer  c  than  the  experimenter's 
guess.  Wetherill  (1963)  showed  that  the  procedure  (2)  with  larger  c  is  less 
susceptible  to  a  poor  choice  of  Xj  especially  for  small  samples. 

Since  the  logit  (and  any  other  parametric)  assumption  is  vulnerable  on 
the  extreme  tails,  it  may  be  desirable  to  use  an  estimation  method  that 
places  less  weights  on  the  observations  with  more  extreme  xVs.  For  data 
generated  by  sequential  procedures  like  (2>,  (4)  and  (5),  the  x^'s  in  the 
initial  runs  tend  to  be  more  extreme.  A  simple  way  to  achieve  this  is  to 
insert  weight  w;  «*  w(|xj-xn|)  on  both  sides  of  <8>  and  solve  iteratively  the 
weighted  version  of  the  likelihood  equation  (8),  where  w<2)  is  decreasing  in 
2  )  0,  and  xn  is  considered  to  be  a  good  estimate  of  Lp.  If  we  choose  w-  to 
be  0  or  1 ,  it  is  equivalent  to  performing  the  unweighted  MLE  based  on  a 
subset  of  data  with  moderate  Xj's.  The  general  question  of  robust  estimation 
for  quanta!  response  data  was  addressed  in  Hiller  and  Kalpern  (1980). 

Let  L*n)  be  the  MLE  of  L_  from  n  observations.  Its  variance  var(L*n>) 
P  P  P 

can  also  be  estimated  via  the  same  parametric  model  by  a  standard  method.  A 
stopping  rule  may  be  devised  based  on  this  variance  estimate.  This  provides 
another  advantage  of  the  parametric  approach  over  the  nonparametr i c  one. 


4.  ft  sequent i al  desion  for  estimating  Lg  j  based  on 

•  • 

the  Spearman-Karber  estimator 

If  the  unknown  response  curve  F(x)  =  H(x-a,f)  is  skew-symmetric  about  a, 
i.e.  N(2,f)  ♦  H(-2  )  =  2H(0,f)  for  any  2,4,  «  is  both  the  median  Lg>5  and 
the  mean  of  F.  The  Spearman-Karber  estimator  (Finney,  1978,  p.  394)  is  a 
nonparametr ic  estimator  of  the  (di sere t i 2ed)  mean  of  F, 
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“SK  •  i,  <5j  -  Sj-l>l<xj-l,xj> 


where  Xj  <  ...  <  Xj,  Hj  observations  are  taken  at  Xj  with  r^  responses,  = 
rj/nj»  n  “  •  Under  conditions  that  ensure  that  ctgK  is  an  efficient 
estimator  of  a,  an  alternative  sequential  design  for  estimating  the  median 
L0  5«a  is  the  following: 


<i)  compute  ccgj^  *  “SK<<yi ,x i *1* » 

(11) 

<ii>  set 


The  two  distinct  advantages  of  the  procedure  (11)  are:  1)  computational 
ease,  2)  weak  assumption  on  F,  i.e.,  the  functional  form  of  H  is  not  assumed 
known.  But  the  price  to  pay  for  these  is  quite  dear.  The  conditions 
required  to  ensure  a  proper  performance  of  (ID  are  quite  restrictive. 
First,  F  should  be  skew-symraetr ic  so  that  its  mean  and  median  are  equal. 
Since  «SK  is  an  unbiased  estimator  of  the  discretized  mean,  not  the 

population  mean,  their  difference  becomes  negligible  only  when  the  spacing 

n  A 

{Xj}j  is  reasonably  dense.  A  proper  use  of  otgg  requires  that  Xj  and  Xj  are 

chosen  such  that  F(xj)  ■  0,  F(Xj)  «  1,  which  may  be  hard  to  achieve  in  the 

initial  stage  of  the  type  of  sequential  designs  considered  in  the  paper.  Jf 

the  experimenter  has  to  pray  for  the  validity  of  these  assumptions,  the 

procedure  (11)  can  not  be  truly  termed  'nonpar ametric.'  Therefore  it  will 

not  be  included  in  the  empirical  study  of  \6. 


5.  Some  theoretical  results  relatino  the  sequential  designs  (5) 
to  the  Stochast i c  Approx imat ion  method. 

Our  efficiency  claim  in  for  the  'logit-MLE'  version  of  (5)  is  based 
on  the  good  faith  in  MLE  for  the  logit  mode).  This  loose  claim  will  be 


reinforced  in  this  section  by  showing  that,  for  the  estimation  of  any 
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percentile  Lp,  a  linear  approximation  to  the  'logit-HLE*  version  of  <5)  is 
equivalent  to  the  asymptotically  fully  efficient  version  <2a>  of  the 
Stochastic  Approximation  scheme.  It  is  interesting  to  note  that,  while  the 
former  is  a  seemingly  parametric  procedure,  the  latter  is  nonparametr i c . 
This  lends  further  support  to  our  previous  argument  that  the  "logic-MLE* 
version  is  not  sensitive  to  distributional  assumption,  at  least  for  the 
middle  percentiles. 

The  following  simple  approximation 


(12)  — It  &  i  *  It 

He  1  2  6 

has  a  maximum  error  of  0.07  in  the  range  1 1 1  4  3  (Cox,  1970,  p.  90).  Assume 
that  |X(Xj~-a>|  4  3  for  all  i  in  the  likelihood  equation  (6).  (This  may  be 
achieved  by  applying  the  MLE  to  the  data  satisfying  the  constraint.)  By 
applying  the  approximation  (12)  to  (8),  we  have 


(13) 


n  .  .  n  n 

£  §  *  £  SUXj-p)  "  I>j»  l»  ■ 

R  Xj  ,n  .  n 

+  dfaxi  "  ‘‘V  “  f'iV 


Let  S  and  p  be  the  solutions  to  (13).  The  estimator  on  of  the  median  Lfl  g  = 
a  is  obtained  as  follows: 


(14) 


n  n  <  n  «n  . 

2  _  2>iZ(yi-|)xj  -  2xf i(yr|) 

C”  c  n  n  .  n  n  . 

*  XiI<>V2)xi  ■ 


which  is  the  weighted  average  of  x.,  i  B  l,...,n  with  weight  w^  proportional 

to  ZOv$>Xj  -  sSti-j-J*  -  i  -  \  y  l 


More  generally  for  any  p,  we  can  consider  the  approximation 


(15) 


j(t)  -  i|-  a  p*<t-j“1<p)>j/<j‘1(p)),  J-1(p)  •  -ln(J  -  !), 

which  is  approximately  valid  -for  Xj  close  to  Lp.  By  applying  (15)  to  (8)  we 
obtain 


n  ,  n 

£aXj-*L  >  ■  - 4 - £(y=-p) 

1  P  J'(J  2<p))  1 


(14) 


ZQx?->l  X,)  - - 4 - £<y,-p>Xj 

•  P  1  J'(J  *(p))  '  ' 


where  the  lOOp  percentile  L  ■  a  -  1 n<p  -!)■«♦  j^J~*(p).  The  estimator 
Lpn>,  (17),  is  obtained  from  *Lp  and  1  by  solving  (16), 


n  ,n 


(17) 


f(n) 

P 


/\  n  n 

*lp  _  Zxi£(yi-p)xi  -  £xf£(y}-p) 

a  n  n  n  n 

>  £i£<yj-p)xi  -  Z*jZ<yj-p) 


which  is  the  weighted  average  of  x  (  with  weight  w^  proportional  to 

^(yj-p)(Xj-x j) .  Since  Lpn*  is  independent  of  J'<J_1(p>>  of  (16),  J'(J-,(p)> 
in  the  approximation  (15)  can  be  replaced  by  any  other  constant  without 
affecting  the  subsequent  results  (17)  and  (18).  Formula  (17)  extends  (14). 
Note  that  some  Wj  may  be  negative.  The  denominators  of  (14)  and  (17) 

n 

are  both  equal  to  n  £  x{  -  £  l£x } t  which  is  nonzero  unless 

y  1*1  y  |— l 

(£  0”1  £  Xj  ■  n"*J>j.  From  (17)  and  after  some  algebras,  it  is  easy  to 

yj-1  )  yj-1 

show  that  the  (nel)^  run,  according  to  the  procedure  (5), 


where  xn  *  n  *£Xj.  Therefor#  the  linear  approximation  (18)  to  our  procedure 
<5)  is  asymptotically  fully  efficient  if  cR  in  (18)  converges  almost  surely 
to  [p'(Lp)j  To  this  end,  note  that  the  regression  slope  estimate  pR  in 
<3>  converges  to  F'(Lp)  a.s.  By  comparing  (18)  and  (3),  cft  -  p”1  *  n(xn  - 

_  2  n 

xn>  /Zx/Xj-Xn).  Since  both  procedures  converge  to  L  for  large  n,  xn  a  xR 
and  cn  -  pn  — *  0  follows  from  the  assumption  F'(Lp)  >  0.  Therefore  the 
asymptotic  full  efficiency  of  (18)  follows  from  similar  results  of  Anbar 
(1978)  and  Lai  and  Robbins  (1981).  (Their  regularity  conditions  do  not  apply 
directly  to  the  quantal  response  problem  but  their  technique  can  be  modified 
to  suit  our  purpose.) 


ilation  stud> 


Under  comparison  are  (i)  the  logit-MLE  version  of  the  sequential  designn 
(3)  with  truncation  as  defined  in  (10)  (abbreviated  as  MLE  in  the  Tables), 
<ii)  the  following  adaptive  Robbins-Monro  (ARM)  design  with  truncation, 


*-n  A-1 
xn*l  "  xn  "  n~<yn~P>*  cn  *  m«n(c,pn,))l 


A 

where  pn  is  defined  in  (3),  (iii)  the  Robbins-Monro  <RM>  design  (2),  and  (iv) 
the  Up-and-Down  (UD)  design  (4). 

Since  the  MLE  of  the  logit  model  does  not  often  exist  for  very  small 
sample  size,  we  fix  an  initial  design  of  size  10  and  a  parametric 
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distribution  H  for  the  quantal  response  curve.  For  each  stimulus  level  x ■  t  i 
*  in  the  initial  design,  ■  0  or  1  is  generated  according  as  U|  } 

or  <  H(x-),  where  is  the  l*b  uniform  random  number  in  CO,  13.  Let 
**10**10  *  bt  the  MLE  of  («,X>  in  the  logit  model  based  on  (X',yj)j°.  The 
common  starting  value  for  all  designs  under  comparison  is  chosen  to  be  Xj}  » 

A  .1 

*10  “  *10  ,n<P  according  to  (9)(ii).  Once  X|j  is  chosen,  the  subsequent 

design  levels  Xjjf.tXgg  are  generated  according  to  different  design 

A  A 

schemes,  but  with  the  same  random  numbers  Uj.  If  the  MLE  (a,  g,Xjp)  does  not 

A  A 

exist  the  simulation  sample  is  discarded.  On  the  other  hand,  if  <o(]o**lO> 
exists  and  is  unique,  the  subsequent  MLE  always  exists  as  is  obvious  from 
condition  (9).  This  is  repeated  for  500  times,  including  those  discarded  due 
to  the  nonexistence  of  MLE.  (The  total  number  of  discarded  samples  is 
denoted  by  M  in  Tables  1  and  II.  The  actual  number  of  simulation  samples  in 
our  study  varies  from  386  to  484.)  For  sample  size  n,  the  Monte  Carlo  mean 
square  error  (MSE)  of  a  sequential  design  is  calculated  as  the  average  of 
<xn  -  Lp>2  over  the  simulation  samples.  In  Table  I,  vmII  are  given  for  the 
desings  (i)  -  (iv)  for  estimating  Lp  5.  In  Table  II,  ^ISE  are  given  for  the 
designs  (i)  -  <iii)  for  estimating  Lq 

For  the  study  of  robustness  to  distributional  assumption  and  to  the 
choice  of  starting  value  and  other  constants,  we  choose  a  variety  of  initial 
designs  and  response  curves  to  reflect  the  degrees  of  the  experimenter's 
knowledge  of  the  response  curve.  (But  note  that  for  design  (i>  the  MLE  is 
always  computed  on  the  assumption  of  the  logit  model  no  matter  what  the  true 
response  curve  is.)  In  Tables  l(a)(c)  and  II(a)(c)(e),  the  standard  logit 
model  is  used  for  the  true  response  curve.  In  other  tables,  the  probit 
models  with  different  locations  and  scales  are  used  for  the  true  response 
curve.  In  each  table  Lp  denotes  the  design  level  that  is  the  lOOp  percentile 
of  the  corresponding  response  curve.  Therefore,  for  example,  the  two  initial 
designs  in  Tables  1(a)  and  1(b)  are  identical,  but  correspond  to  different 
percentiles  under  different  response  curves. 
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Tab)*  1.  Moat r  Carlo  "fiil  <Root  Mean  Square  Error)  o4  Sequential  Designs  lor 
Est  taating  the  SO  Percent  i  le  of  the  Trot  Ouantal  Response  Cum 


Initial  design: 


stiaulus  level 


l0.1  l0.3 


ao.  of  observations  1  2  4 

True  response  curve:  logit  aodel  (d)  with  #  =  0,1=1 


.5  l0.7 
4  2 


no.  truncations 


12 

.72 

.50 

. 

18 

.53 

.47 

. 

0 

.49 

.40 

.< 

1 

.54 

.48 

. 

'8 

.50 

.45 

.< 

>9 

1.09 

.92 

of 

L  «  INp  percentile  of  the  true  response  curve, 

HLE-d  ■  procedure  <10)  with  truncation  constant  d 
AM-c  «  procedure  (19)  wi th  truncation  constant  c 
RH-c  ■  procedure  <2)  with  constant  c 
UthA  •  procedure  (4)  with  coni  tint  i 

no.  truncations  «  total  nunber  of  truncations  of  the  kind  (10)  or  (19) 
N  *  total  nmber  of  simulation  samples  for  which  no  MLE  exists 


Initial  design  (sane  as  1(a)): 


stiaulus  level  L,#3  L,^  l0iM  Lp>w 


no.  of  observations  12421 
True  response  curve:  probit  aodel  (7)  with  p  ■  -0.5,  i  «  3.1915 


no.  truncations 


1.34 

1.07 

.85 

.82 

.77 

859 

1.10 

.88 

.83 

.77 

.73 

472 

.93 

.85 

.74 

.83 

.02 

214 

.90 

.79 

.71 

.84 

.03 

02 

1.13 

.87 

.75 

.70 

.03 

4 

1.70 

1.54 

1.30 

1.21 

1.08 

1411 

1.58 

1.42 

1.23 

1.08 

.90 

025 

1.02 

1.39 

1.21 

1.07 

.92 

251 

1.80 

1.50 

1.32 

1.19 

1.02 

90 

3.32 

2.90 

2.37 

1.80 

1.50 

9 

1.77 

1.51 

1.35 

1.17 

1.05 

1.09 

1.47 

1.28 

1.10 

.93 

1.92 

1.81 

1.71 

1.03 

1.50 

2.22 

2.21 

2.27 

2.18 

2.15 

1.73 

1.58 

1.00 

1.48 

1.48 

1.83 

1.04 

1.40 

1.31 

1.13 

H» 

50 

• 
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Table  II.  Monte  Carlo  ^tS£  (Root  Mean  Square  Error)  of  Sequential  Design*  for 
Estimating  the  75  Percentile  of  the  True  Ouantal  Response  Curve 


Initial  design  and  true  response  curve:  same  as  in  1(a) 


design 


HLE-30 

MLE-50 

HLE-100 

HLE-200 

MLE-600 

ARM-14 

ARM-30 

ARM-50 

AftHOO 

ARM-400 

RM-32 

RM-14 

m-4 


no.  truncations 


K. 


<bl) 


Initial  design : 


(tiaulat  'ml*  L#J  LM<  L,w  Lfl^  L#  M 

3  6  3  1 


•o.  tri  observations  } 
Initial  smple  size  H 
True  rtspontr  curve:  sane  as  in  1Kb) 


R 


design 

14 

20 

25 

30 

35 

no.  truncations 

MLE-30 

2.23 

1.43 

1.37 

1.13 

.99 

2494 

MLE-50 

2.23 

1.42 

1.20 

1.04 

.97 

1387 

HLE-100 

2.27 

1.31 

1.14 

1.07 

1.01 

483 

MLE-200 

2.74 

1.40 

1.18 

1.11 

1.03 

140 

MLE-400 

2.91 

1.48 

1.17 

1.23 

1.07 

32 

AAH-16 

2.24 

1.97 

1.47 

1.48 

1.31 

2114 

AM-30 

2.24 

1.91 

1.40 

1.44 

1.31 

B55 

ASH-50 

2.24 

1.97 

1.71 

1.32 

1.38 

388 

AM-100 

2.31 

2.10 

1.82 

1.42 

1.41 

113 

AM-400 

2.94 

3.34 

3/12 

2.72 

2.38 

11 

M-32 

2.13 

1.47 

1.34 

1.20 

1.04 

M-14 

2.20 

1.77 

1.44 

1.28 

1.13 

M-4 

2.31 

2.13 

2.01 

1.90 

1.81 

N>  14 

•  sane  as 

in  IKb) 

<C) 

Initial  design  and  true  response  cureei 

sane  as 

in  1(c) 

design 

12 

14 

n 

20 

23 

30 

35 

no.  truncations 

MLE-30 

2.38 

1.94 

1.37 

1.28 

1.08 

.93 

800 

MLE-30 

2.44 

1.37 

1.15 

.83 

.58 

.44 

494 

MLE-100 

2.22 

1.02 

.34 

.34 

.44 

.43 

184 

MLE-200 

1.89 

.83 

.41 

.52 

.45 

.41 

82 

MLE-400 

1.44 

.83 

.57 

.54 

.44 

.47 

19 

AM-30 

2.43 

2.12 

1.75 

1.47 

1.24 

1.11 

330 

AM-30 

2.41 

1.93 

1.33 

1.18 

.94 

.82 

204 

AM-100 

2.42 

1.74 

1.34 

1.09 

.93 

.87 

82 

AM-200 

2.72 

1.81 

1.54 

1.23 

1.03 

.94 

13 

AM-400 

2.73 

1.88 

1.84 

1.49 

1.17 

1.03 

1 

M-32 

2.77 

2.13 

1.79 

1.42 

1.24 

1.07 

M-14 

2.71 

2.32 

2.08 

1.84 

1.44 

1.53 

M-4 

2.78 

2.45 

2.54 

2.47 

2.40 

2.33 

«■  99 
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<d) 


Initial  design  and  true  response  curve:  sine  as  in  1(d) 


30 


35 


no.  truncations 
782  ”” 


a 


design 

12 

14 

20 

25 

30 

35 

HLE-30 

2.5? 

1.72 

1.14 

.81 

.4? 

.40 

HLE-50 

2.44 

1.1? 

.82 

.40 

.46 

.41 

HLE-100 

2.14 

.?1 

.57 

.4? 

.44 

.41 

HLE-200 

1.42 

.84 

.40 

.4? 

.45 

.42 

HLE-600 

1.24 

.77 

.40 

.50 

.44 

.43 

AB1-30 

2.44 

1 .93 

1.42 

1.08 

.?3 

.81 

ASN-50 

2.40 

1.74 

1.2? 

.?8 

.82 

.70 

AKfrlOO 

2.47 

1.78 

1.30 

1.01 

.88 

.77 

AW-200 

2. 63 

2.03 

1.4? 

1.1? 

.84 

ARM-400 

2.83 

2.48 

l.?2 

1.54 

1.30 

1.12 

RH-32 

2.74 

1.8? 

1.3? 

1.03 

.?4 

.84 

m-i4 

2.72 

2.23 

1.87 

1.55 

1.31 

1.11 

RH-4 

2.82 

2.45 

2.53 

2.42 

2.33 

2.24 

H*  71 


(e) 


stimulus  level 

Initial  design: 

no.  of  observations 


L0.2  L0 .3  L0.5  L0.7 
3  3  2  1 


True  response  curve:  logit  nodel  (4)  with  a  ■  0,  \  *  1 


420 

153 

54 

7 

233 

83 

44 

5 

1 


n 


design 

12 

14 

20 

25 

30 

35 

no.  truncations 

HLE-30 

4.44 

3.55 

2.98 

2.44 

2.01 

1.48 

1940 

HLE-50 

4.53 

2.?8 

2.11 

1.34 

.94 

.67 

1178 

MLE-180 

4.34 

1.71 

.87 

.71 

.44 

.57 

417 

HLE-200 

4.14 

1.88 

.?0 

.73 

.70 

.43 

143 

HLE-400 

3.45 

1.92 

.92 

.78 

.80 

.44 

40 

ARM-30 

4.44 

3.84 

3.38 

2.97 

2.44 

2.41 

246 

Aftf-50 

4.45 

3.85 

3.34 

2.95 

2.43 

2.38 

125 

AHM-100 

4.48 

3.?2 

3.43 

3.01 

2.48 

2.42 

35 

ARM-200 

4.81 

4.08 

3.54 

3.12 

2.78 

2.50 

5 

ARM-481 

4.81 

4.31 

3.4? 

3.18 

2.7? 

2.51 

1 

RH-32 

4.51 

3.55 

2.97 

2.38 

1.97 

1.44 

04-14 

4.44 

4.81 

3.45 

3.33 

3.08 

2.89 

01-4 

4.83 

4.42 

4.47 

4.33 

4.23 

4.14 

H  ■ 

94 

(4) 

stimulus  level 

Initial  design  (saie  as  11(e)): 

L0.15  4.2? 

4.42  4.62 

4.80 

no.  of  observations 

1  3 

3  2 

1 

True  response  curve:  probit  nodel  (7)  with  p  ■  -1.5, 

e  ■  1.5957 

design 

12 

14 

20 

a 

25 

30 

35 

no. ‘truncations 

The  results  in  Tables  I  and  11  are  summarized  in  the  'following. 


<A)  General  compar i son  of  designs. 

In  general,  the  performance  of  the  designs  is  in  the  following 
descending  order, 


MLE  >  ARM  >  RM  >  UD 

Only  in  Table  1 1  <  b>  does  RM-16  (the  Robbins-Monro  method  (2)  with  c  *  16) 
outperform  the  others.  Gut  when  we  increase  the  size  of  the  initial  design 
from  10  to  14  as  in  Table  II(bi),  MLE  has  again  the  best  performance. 

Uithin  RM  we  observe  the  descending  order  of  performance 

RM-32  and  RM-16  >  RM-4  >  RM-1  >  RM-0.25. 

For  Tables  l(a)(b)(d)  and  IKaXb),  RM-16  >  RM-32;  for  other  tables,  RM-32  > 

RM-1 6.  Note  that  RM-4  is  asymptotically  fully  efficient  if  the  response 

curve  is  the  standard  logit  (as  assumed  in  Tables  I(aXc>,  IKaXcXe)) 

because  F'(0)  ■  *.  RM-4  certainly  fails  to  deliver  this  asymptotic  promise 
4 

of  optimality  for  n  as  large  as  35.  Asymptotic  results  seem  quite  irrelevant 
in  this  context.  Uithin  UD,  we  observe  the  descending  order  of  performance 

UD-0.25  >  UD-1  >  UD-2  >  UD-4, 

where  UD-A  means  the  Up-and-Down  method  (4)  with  step  size  A.  To  save  space, 
RM-1,  RM-0.25  and  UD-4  are  not  included  in  the  tables. 

Since  our  interest  is  in  finding  superior  designs,  we  will  confine  the 
remaining  discussion  to  MLE,  ARM,  RM-32  and  RM-16.  A  very  complete 
comparison  of  the  empirical  performance  of  RM-c  and  UD-A  for  different  c  and 
A  was  done  in  Uetherill  (1963). 

<B)  Superior i ty  of  the  looi t-MLE  desion. 

The  superiority  of  the  logit-MLE  design  (10)  with  truncation  constant  d, 
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hereafter  denoted  as  MLE-d,  is  broad-based.  In  the  eleven  tables,  MLE-50, 
MLE-100,  MLE-200  consistently  outperform  the  best  ARM.  Except  in  Table 
11(c),  MLE-30  outperforms  the  best  ARM.  The  efficiency  gain  of  MLE  over  ARM 
is  more  conspicuous  for  larger  n. 

Uhat  truncation  constant  d  should  be  chosen?  The  MLE  designs  with  50  { 
d  4  200  all  perform  well.  Within  this  range  their  difference  of  performance 
is  probably  negligible.  MLE-30  does  not  perform  as  well,  because  a  forceful 
truncation  like  d  -  30  limits  the  potential  of  the  MLE  design  in  making  more 
flexible  and  justifiably  large  moves  when  the  design  levels  are  not  yet  close 
enough  to  the  target  value.  On  the  other  hand,  the  performance  of  MLE-600, 
which  involves  very  weak  truncations,  is  more  fluctuating.  For  n  \  20, 
MLE-600  is  comparable  to  the  best  MLE  design.  For  small  n,  MLE-600  is 
comparable  to  the  best  MLE  design  in  Tables  I(a)(b)  and  Il(a)(b),  but  worse 
than  MLE-50,  MLE-100,  and  MLE-200  in  Tables  I<c)(d)  and  II(bl).  In  Tables 
IKcXdXeXf ) ,  MLE-600  is  much  better  than  the  other  MLE  designs  for  n  »  12 
(an  uninteresting  case),  and  is  comparable  to  MLE-200  for  n  >  16.  For  n  l 
20,  the  effect  of  truncation  is  negligible  over  100  4  d  4  400. 

Since  a  major  purpose  for  finding  better  designs  is  to  reduce  the  number 
of  runs  required  for  satisfying  an  error  bound,  we  shall  measure  the 
efficiency  gain  of  the  MLE-design  over  the  ARM  design  by  such  numbers.  In 
each  case,  we  find  the  smallest  achieved  by  the  best  ARM  design  at  n  B 
35.  We  then  find  m  to  be  the  smal  1  est  sample  size  at  which  an  MLE  desion 
ach i eves  the  same  ^MSE .  In  Table  III,  the  values  of  m  are  obtained  by  linear 
interpolation  for  the  eleven  tables  in  Tables  I  and  II. 


Table  III.  Values  of  m  for  Tables  I(a)-(d),  IKa)r(f) 


26  20  18  25 


29  17 


15  14 


The  percentage  of  runs  saved  by  using  the  best  MLE  design  instead  of  the  best 
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ARM  design  ranges  ■front  25%  to  60%.  This  surprising  difference  of  performance 
can  be  explained  by  the  different  natures  of  the  two  approximation  schemes. 
The  adaptive  Robb i ns-Monro  design  is  a  stochastic  Newton-Raphson  method  which 
uses  1 t near  approximation  to  nonlinear  equation  for  the  iterative  solution  of 
nonlinear  equation.  It  is  Known  to  be  unstable  unless  the  starting  value  is 
close  to  the  target  value.  Only  under  this  premise  does  the  large  sample 
results  like  asymptotic  normality  and  efficiency  make  sense  for  small  or 
moderate  samples.  On  the  other  hand  our  logit-MLE  method  seems  to  be  free 
from  this  problem  since  a  s i gmo i d  curve  is  used  in  the  iterative  solution  of 
nonlinear  equation. 

C.  Improvement  of  ARM  over  RM. 

There  is  a  slight  but  definite  improvement  of  ARM-c  (procedure  (19)  with 
truncation  constant  c)  with  c  *  SO,  100  over  the  best  RM  in  Tables  KaXc) 
IKcXd).  The  best  RM  design  (RM-16  or  RM-32)  is  usually  quite  comparable  to 
the  best  ARM  design.  In  Tables  1 1  (aXbXbl  XeXf  >  it  even  beats  the  best 
ARM.  (But  RM-4  is  definitely  inferior  to  the  best  ARM>) 

The  best  performance  of  ARM  occurs  with  ARM-c  with  16  <  c  <  100  with  the 
majority  of  them  in  the  narrower  range  30  {  c  <50.  The  ARM-600  design, 
which  involves  very  weak  truncation,  is  a  real  disappointment.  Except  for 
Table  11(c),  it  is  worse  than  the  best  (nonadaptive)  RM  design.  It  is 
consistently  worse  than  the  best  ARM  design,  and  for  Tables  Il(bXblXeXf) 
much  worse.  Asymptotic  full  efficiency  is  a  quite  irrevelant  concept  here. 
Moreover  the  MSE  of  ARM-600  exhibits  an  erratic  pattern,  e.g,  it  sometimes 
increases  as  n  increases.  Generally  the  ARM  requires  more  severe  truncation 
than  the  MLE.  This  is  because  the  ARM  can  make  an  unduly  large  move  from  xn 
to  xn4]  as  explained  in  Section  2. 


D. 


In  Tables  I  and  II  we  have  counted  the  total  number  of  times  the 


truncation  (defined  in  <10)  and  (19>>  ie  invoked.  For  the  same  truncation 
constant,  the  MLE  design  always  requires  more  trunctations  than  the  ARM 
Design.  It  suggests  that  the  MLE  design  makes  large  moves  more  frequently 
than  the  ARM  design.  Since  MLE-100,  MLE-200  and  MLE-600  do  very  well  in  the 
study,  such  large  moves  are  probably  justified. 

Ule  have  also  examined  the  empirical  behavior  of  the  same  set  of  designs 
for  initial  designs  of  size  25.  The  results  are  very  similar.  As  the  size 
of  the  initial  design  increases,  the  number  of  simulation  samples  for  which 
no  MLE  exists  quickly  drops. 

Since  the  MLE-d  designs  with  50  {  d  4  6 00  perform  extremely  well  in  a 
variety  of  situations  considered  in  this  paper,  we  suggest  that  they  may  be 
considered  seriously  in  practical  work. 
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